Identifying medical terms in patient-authored text: a crowdsourcing-based approach
نویسندگان
چکیده
BACKGROUND AND OBJECTIVE As people increasingly engage in online health-seeking behavior and contribute to health-oriented websites, the volume of medical text authored by patients and other medical novices grows rapidly. However, we lack an effective method for automatically identifying medical terms in patient-authored text (PAT). We demonstrate that crowdsourcing PAT medical term identification tasks to non-experts is a viable method for creating large, accurately-labeled PAT datasets; moreover, such datasets can be used to train classifiers that outperform existing medical term identification tools. MATERIALS AND METHODS To evaluate the viability of using non-expert crowds to label PAT, we compare expert (registered nurses) and non-expert (Amazon Mechanical Turk workers; Turkers) responses to a PAT medical term identification task. Next, we build a crowd-labeled dataset comprising 10 000 sentences from MedHelp. We train two models on this dataset and evaluate their performance, as well as that of MetaMap, Open Biomedical Annotator (OBA), and NaCTeM's TerMINE, against two gold standard datasets: one from MedHelp and the other from CureTogether. RESULTS When aggregated according to a corroborative voting policy, Turker responses predict expert responses with an F1 score of 84%. A conditional random field (CRF) trained on 10 000 crowd-labeled MedHelp sentences achieves an F1 score of 78% against the CureTogether gold standard, widely outperforming OBA (47%), TerMINE (43%), and MetaMap (39%). A failure analysis of the CRF suggests that misclassified terms are likely to be either generic or rare. CONCLUSIONS Our results show that combining statistical models sensitive to sentence-level context with crowd-labeled data is a scalable and effective technique for automatically identifying medical terms in PAT.
منابع مشابه
PROVIDE A MODEL FOR IDENTIFYING AND RANKING THE MANAGERIAL FACTORS AFFECTING INFORMATION SECURITY IN ORGANIZATION BY USING VIKOR METHOD; CASE STUDY: TEHRAN UNIVERSITY OF MEDICAL SCIENCES
<span style="color: #000000; font-family: Tahoma, sans-serif; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline !important; float: none; ba...
متن کاملPROVIDE A MODEL FOR IDENTIFYING AND RANKING THE MANAGERIAL FACTORS AFFECTING INFORMATION SECURITY IN ORGANIZATION BY USING VIKOR METHOD; CASE STUDY: TEHRAN UNIVERSITY OF MEDICAL SCIENCES
<span style="color: #000000; font-family: Tahoma, sans-serif; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline !important; float: none; ba...
متن کاملIdentifying the effective factors in using medical evidence in clinical medicine: A qualitative study in the emergency department
Introduction: Evidence-based medicine can increase the clinical performance of physicians. This qualitative study was conducted with the aim of examining the effective factors for using new evidence by emergency medicine residents in Rasoul Akram and Haftam Tir Hospitals in Tehran, Iran. Methods: The present descriptive-analytical study was conducted in the year 2016 in two hospitals In Tehran...
متن کاملبررسی مستندات منشور حقوق بیمار بر مبنای آموزههای دینی (قرآن و روایات)
Today, paying attention to patients' rights and their satisfaction with quality and medical care is one of the most important priorities. Observing patient's rights, informing them and sharing them in decision-making, is greatly effective in improving patients. Hence, Iran's health care system has made a decree to this end. The purpose of the present paper is to analyze the extent to which the ...
متن کاملIdentifying Indicators Affecting the Evaluation of Service Quality of Medical Centers’ Online Appointment Systems
Introduction: Online queuing systems in medical centers significantly reduce waiting time and costs, and increase patient satisfaction with the quality of services provided. Service provisions with the desired quality through these systems will manage the crowds in the health care centers. In the current situation, gatherings cause an upward trend of the COVID-19 pandemic and subsequent problem...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره 20 شماره
صفحات -
تاریخ انتشار 2013